Modeling the multi-state natural history of rare diseases with heterogeneous individual patient data: A simulation study.

Multi-state survival models are used to represent the natural history of a disease, forming the basis of a health technology assessment comparing a novel treatment to current practice. Constructing such models for rare diseases is problematic, since evidence sources are typically much sparser and more heterogeneous. This simulation study investigated different one-stage and two-stage approaches to meta-analyzing individual patient data (IPD) in a multi-state survival setting when the number and size of studies being meta-analyzed are small. The objective was to assess methods of different complexity to see when they are accurate, when they are inaccurate and when they struggle to converge due to the sparsity of data. Biologically plausible multi-state IPD were simulated from study- and transition-specific hazard functions. One-stage frailty and two-stage stratified models were estimated, and compared to a base case model that did not account for study heterogeneity. Convergence and the bias/coverage of population-level transition probabilities to, and lengths of stay in, each state were used to assess model performance. A real-world application to Duchenne Muscular Dystrophy, a neuromuscular rare disease, was conducted, and a software demonstration is provided. Models not accounting for study heterogeneity were consistently out-performed by two-stage models. Frailty models struggled to converge, particularly in scenarios of low heterogeneity, and predictions from models that did converge were also subject to bias. Stratified models may be better suited to meta-analyzing disparate sources of IPD in rare disease natural history/economic modeling, as they converge more consistently and produce less biased predictions of lengths of stay.

Exploring different research questions via complex multi-state models when using registry-based repeated prescriptions of antidepressants in women with breast cancer and a matched population comparison group.

BACKGROUND: Multi-state models are used to study several clinically meaningful research questions. Depending on the research question of interest and the information contained in the data, different multi-state structures and modelling choices can be applied. We aim to explore different research questions using a series of multi-state models of increasing complexity when studying repeated prescriptions data, while also evaluating different modelling choices. METHODS: We develop a series of research questions regarding the probability of being under antidepressant medication across time using multi-state models, among Swedish women diagnosed with breast cancer (n = 18,313) and an age-matched population comparison group of cancer-free women (n = 92,454) using a register-based database (Breast Cancer Data Base Sweden 2.0). Research questions were formulated ranging from simple to more composite ones. Depending on the research question, multi-state models were built with structures ranging from simpler ones, like single-event survival analysis and competing risks, up to complex bidirectional and recurrent multi-state structures that take into account the recurring start and stop of medication. We also investigate modelling choices, such as choosing a time-scale for the transition rates and borrowing information across transitions. RESULTS: Each structure has its own utility and answers a specific research question. However, the more complex structures (bidirectional, recurrent) enable accounting for the intermittent nature of prescribed medication data. These structures deliver estimates of the probability of being under medication and total time spent under medication over the follow-up period. Sensitivity analyses over different definitions of the medication cycle and different choices of timescale when modelling the transition intensity rates show that the estimates of total probabilities of being in a medication cycle over follow-up derived from the complex structures are quite stable. CONCLUSIONS: Each research question requires the definition of an appropriate multi-state structure, with more composite ones requiring such an increase in the complexity of the multi-state structure. When a research question is related with an outcome of interest that repeatedly changes over time, such as the medication status based on prescribed medication, the use of novel multi-state models of adequate complexity coupled with sensible modelling choices can successfully address composite, more realistic research questions.

Relaxing the assumption of constant transition rates in a multi-state model in hospital epidemiology.

BACKGROUND: Multi-state models are being increasingly used to capture complex disease pathways. The convenient formula of the exponential multi-state model can facilitate a quick and accessible understanding of the data. However, assuming time constant transition rates is not always plausible. On the other hand, obtaining predictions from a fitted model with time-dependent transitions can be challenging. One proposed solution is to utilise a general simulation algorithm to calculate predictions from a fitted multi-state model. METHODS: Predictions obtained from an exponential multi-state model were compared to those obtained from two different parametric models and to non-parametric Aalen-Johansen estimates. The first comparative approach fitted a multi-state model with transition-specific distributions, chosen separately based on the Akaike Information Criterion. The second approach was a Royston-Parmar multi-state model with 4 degrees of freedom, which was chosen as a reference model flexible enough to capture complex hazard shapes. All quantities were obtained analytically for the exponential and Aalen-Johansen approaches. The transition rates for the two comparative approaches were also obtained analytically, while all other quantities were obtained from the fitted models via a general simulation algorithm. Metrics investigated were: transition probabilities, attributable mortality (AM), population attributable fraction (PAF) and expected length of stay. This work was performed on previously analysed hospital acquired infection (HAI) data. By definition, a HAI takes three days to develop and therefore selected metrics were also predicted from time 3 (delayed entry). RESULTS: Despite clear deviations from the constant transition rates assumption, the empirical estimates of the transition probabilities were approximated reasonably well by the exponential model. However, functions of the transition probabilities, e.g. AM and PAF, were not well approximated and the comparative models offered considerable improvements for these metrics. They also provided consistent predictions with the empirical estimates in the case of delayed entry time, unlike the exponential model. CONCLUSION: We conclude that methods and software are readily available for obtaining predictions from multi-state models that do not assume constant transition rates. The multistate package in Stata facilitates a range of predictions with confidence intervals, which can provide a more comprehensive understanding of the data. User-friendly code is provided.

Dynamic assessment of venous thromboembolism risk in patients with cancer by longitudinal D-Dimer analysis: A prospective study.

BACKGROUND: Venous thromboembolism (VTE) is a frequent complication of cancer. Elevated D-dimer is associated with an increased risk of cancer-associated VTE. Whether changes in D-dimer over time harbor additional prognostic information that may be exploited clinically for dynamic prediction of VTE is unclear. OBJECTIVES: To explore the potential role of longitudinal D-dimer trajectories for personalized prediction of cancer-associated VTE. PATIENTS/METHODS: A total of 167 patients with active malignancy were prospectively enrolled (gastrointestinal: n = 59 [35%], lung: n = 56 [34%], brain: n = 50 [30%], others: n = 2 [1%]; metastatic disease: n = 74 [44%]). D-dimer (median = 0.8 µg/mL [25th-75th percentile: 0.4-2.0]) was measured at baseline and during 602 monthly follow-up visits. Joint models of longitudinal and time-to-event data were implemented to quantify the association between D-dimer trajectories and prospective risk of VTE. RESULTS: VTE occurred in 20 patients (250-day VTE risk = 12.1%, 95% confidence interval [CI], 7.8-18.5). D-dimer increased by 34%/month (0.47 µg/mL/month, 95% CI, 0.22-0.72, P < .0001) in patients who developed VTE, but remained constant in patients who did not develop VTE (change/month = -0.06 µg/mL, 95% CI, -0.15 to 0.02, P = .121). In joint modeling, a doubling of the D-dimer trajectory was associated with a 2.8-fold increase in the risk of VTE (hazard ratio = 2.78, 95% CI, 1.69-4.58, P < .0001). This finding was independent of established VTE risk factors. Highly personalized, dynamic predictions of VTE conditional on individual patients' D-dimer trajectories could be obtained. CONCLUSIONS: D-dimer increases before the onset of cancer-associated VTE, but remains constant over time in patients without VTE. This study represents proof-of-concept that longitudinal trajectories of D-Dimer may advance the personalized assessment of VTE risk in the oncologic setting.

Mixed-effects models for health care longitudinal data with an informative visiting process: A Monte Carlo simulation study.

Electronic health records are being increasingly used in medical research to answer more relevant and detailed clinical questions; however, they pose new and significant methodological challenges. For instance, observation times are likely correlated with the underlying disease severity: Patients with worse conditions utilise health care more and may have worse biomarker values recorded. Traditional methods for analysing longitudinal data assume independence between observation times and disease severity; yet, with health care data, such assumptions unlikely hold. Through Monte Carlo simulation, we compare different analytical approaches proposed to account for an informative visiting process to assess whether they lead to unbiased results. Furthermore, we formalise a joint model for the observation process and the longitudinal outcome within an extended joint modelling framework. We illustrate our results using data from a pragmatic trial on enhanced care for individuals with chronic kidney disease, and we introduce user-friendly software that can be used to fit the joint model for the observation process and a longitudinal outcome.

Impact of model misspecification in shared frailty survival models.

Survival models incorporating random effects to account for unmeasured heterogeneity are being increasingly used in biostatistical and applied research. Specifically, unmeasured covariates whose lack of inclusion in the model would lead to biased, inefficient results are commonly modeled by including a subject-specific (or cluster-specific) frailty term that follows a given distribution (eg, gamma or lognormal). Despite that, in the context of parametric frailty models, little is known about the impact of misspecifying the baseline hazard or the frailty distribution or both. Therefore, our aim is to quantify the impact of such misspecification in a wide variety of clinically plausible scenarios via Monte Carlo simulation, using open-source software readily available to applied researchers. We generate clustered survival data assuming various baseline hazard functions, including mixture distributions with turning points, and assess the impact of sample size, variance of the frailty, baseline hazard function, and frailty distribution. Models compared include standard parametric distributions and more flexible spline-based approaches; we also included semiparametric Cox models. The resulting bias can be clinically relevant. In conclusion, we highlight the importance of fitting models that are flexible enough and the importance of assessing model fit. We illustrate our conclusions with two applications using data on diabetic retinopathy and bladder cancer. Our results show the importance of assessing model fit with respect to the baseline hazard function and the distribution of the frailty: misspecifying the former leads to biased relative and absolute risk estimates, whereas misspecifying the latter affects absolute risk estimates and measures of heterogeneity.

Using simulation studies to evaluate statistical methods.

Simulation studies are computer experiments that involve creating data by pseudo-random sampling. A key strength of simulation studies is the ability to understand the behavior of statistical methods because some "truth" (usually some parameter/s of interest) is known from the process of generating the data. This allows us to consider properties of methods, such as bias. While widely used, simulation studies are often poorly designed, analyzed, and reported. This tutorial outlines the rationale for using simulation studies and offers guidance for design, execution, analysis, reporting, and presentation. In particular, this tutorial provides a structured approach for planning and reporting simulation studies, which involves defining aims, data-generating mechanisms, estimands, methods, and performance measures ("ADEMP"); coherent terminology for simulation studies; guidance on coding simulation studies; a critical discussion of key performance measures and their estimation; guidance on structuring tabular and graphical presentation of results; and new graphical presentations. With a view to describing recent practice, we review 100 articles taken from Volume 34 of Statistics in Medicine, which included at least one simulation study and identify areas for improvement.

Assessing methods for dealing with treatment switching in clinical trials: A follow-up simulation study.

When patients randomised to the control group of a randomised controlled trial are allowed to switch onto the experimental treatment, intention-to-treat analyses of the treatment effect are confounded because the separation of randomised groups is lost. Previous research has investigated statistical methods that aim to estimate the treatment effect that would have been observed had this treatment switching not occurred and has demonstrated their performance in a limited set of scenarios. Here, we investigate these methods in a new range of realistic scenarios, allowing conclusions to be made based upon a broader evidence base. We simulated randomised controlled trials incorporating prognosis-related treatment switching and investigated the impact of sample size, reduced switching proportions, disease severity, and alternative data-generating models on the performance of adjustment methods, assessed through a comparison of bias, mean squared error, and coverage, related to the estimation of true restricted mean survival in the absence of switching in the control group. Rank preserving structural failure time models, inverse probability of censoring weights, and two-stage methods consistently produced less bias than the intention-to-treat analysis. The switching proportion was confirmed to be a key determinant of bias: sample size and censoring proportion were relatively less important. It is critical to determine the size of the treatment effect in terms of an acceleration factor (rather than a hazard ratio) to provide information on the likely bias associated with rank-preserving structural failure time model adjustments. In general, inverse probability of censoring weight methods are more volatile than other adjustment methods.

Adjusting Expected Mortality Rates Using Information From a Control Population: An Example Using Socioeconomic Status.

Expected or reference mortality rates are commonly used in the calculation of measures such as relative survival in population-based cancer survival studies and standardized mortality ratios. These expected rates are usually presented according to age, sex, and calendar year. In certain situations, stratification of expected rates by other factors is required to avoid potential bias if interest lies in quantifying measures according to such factors as, for example, socioeconomic status. If data are not available on a population level, information from a control population could be used to adjust expected rates. We have presented two approaches for adjusting expected mortality rates using information from a control population: a Poisson generalized linear model and a flexible parametric survival model. We used a control group from BCBaSe-a register-based, matched breast cancer cohort in Sweden with diagnoses between 1992 and 2012-to illustrate the two methods using socioeconomic status as a risk factor of interest. Results showed that Poisson and flexible parametric survival approaches estimate similar adjusted mortality rates according to socioeconomic status. Additional uncertainty involved in the methods to estimate stratified, expected mortality rates described in this study can be accounted for using a parametric bootstrap, but this might make little difference if using a large control population.

Parametric multistate survival models: Flexible modelling allowing transition-specific distributions with application to estimating clinically useful measures of effect differences.

Multistate models are increasingly being used to model complex disease profiles. By modelling transitions between disease states, accounting for competing events at each transition, we can gain a much richer understanding of patient trajectories and how risk factors impact over the entire disease pathway. In this article, we concentrate on parametric multistate models, both Markov and semi-Markov, and develop a flexible framework where each transition can be specified by a variety of parametric models including exponential, Weibull, Gompertz, Royston-Parmar proportional hazards models or log-logistic, log-normal, generalised gamma accelerated failure time models, possibly sharing parameters across transitions. We also extend the framework to allow time-dependent effects. We then use an efficient and generalisable simulation method to calculate transition probabilities from any fitted multistate model, and show how it facilitates the simple calculation of clinically useful measures, such as expected length of stay in each state, and differences and ratios of proportion within each state as a function of time, for specific covariate patterns. We illustrate our methods using a dataset of patients with primary breast cancer. User-friendly Stata software is provided.

Joint modelling of longitudinal and survival data: incorporating delayed entry and an assessment of model misspecification.

A now common goal in medical research is to investigate the inter-relationships between a repeatedly measured biomarker, measured with error, and the time to an event of interest. This form of question can be tackled with a joint longitudinal-survival model, with the most common approach combining a longitudinal mixed effects model with a proportional hazards survival model, where the models are linked through shared random effects. In this article, we look at incorporating delayed entry (left truncation), which has received relatively little attention. The extension to delayed entry requires a second set of numerical integration, beyond that required in a standard joint model. We therefore implement two sets of fully adaptive Gauss-Hermite quadrature with nested Gauss-Kronrod quadrature (to allow time-dependent association structures), conducted simultaneously, to evaluate the likelihood. We evaluate fully adaptive quadrature compared with previously proposed non-adaptive quadrature through a simulation study, showing substantial improvements, both in terms of minimising bias and reducing computation time. We further investigate, through simulation, the consequences of misspecifying the longitudinal trajectory and its impact on estimates of association. Our scenarios showed the current value association structure to be very robust, compared with the rate of change that we found to be highly sensitive showing that assuming a simpler trend when the truth is more complex can lead to substantial bias. With emphasis on flexible parametric approaches, we generalise previous models by proposing the use of polynomials or splines to capture the longitudinal trend and restricted cubic splines to model the baseline log hazard function. The methods are illustrated on a dataset of breast cancer patients, modelling mammographic density jointly with survival, where we show how to incorporate density measurements prior to the at-risk period, to make use of all the available information. User-friendly Stata software is provided.

Adjusting survival time estimates to account for treatment switching in randomized controlled trials--an economic evaluation context: methods, limitations, and recommendations.

BACKGROUND: Treatment switching commonly occurs in clinical trials of novel interventions in the advanced or metastatic cancer setting. However, methods to adjust for switching have been used inconsistently and potentially inappropriately in health technology assessments (HTAs). OBJECTIVE: We present recommendations on the use of methods to adjust survival estimates in the presence of treatment switching in the context of economic evaluations. METHODS: We provide background on the treatment switching issue and summarize methods used to adjust for it in HTAs. We discuss the assumptions and limitations associated with adjustment methods and draw on results of a simulation study to make recommendations on their use. RESULTS: We demonstrate that methods used to adjust for treatment switching have important limitations and often produce bias in realistic scenarios. We present an analysis framework that aims to increase the probability that suitable adjustment methods can be identified on a case-by-case basis. We recommend that the characteristics of clinical trials, and the treatment switching mechanism observed within them, should be considered alongside the key assumptions of the adjustment methods. Key assumptions include the "no unmeasured confounders" assumption associated with the inverse probability of censoring weights (IPCW) method and the "common treatment effect" assumption associated with the rank preserving structural failure time model (RPSFTM). CONCLUSIONS: The limitations associated with switching adjustment methods such as the RPSFTM and IPCW mean that they are appropriate in different scenarios. In some scenarios, both methods may be prone to bias; "2-stage" methods should be considered, and intention-to-treat analyses may sometimes produce the least bias. The data requirements of adjustment methods also have important implications for clinical trialists.